Viewpoint-Free Photography for Virtual Reality

Viewpoint-free photography, i.e., interactively controlling the viewpoint of a photograph after capture, is a standing challenge. In this thesis, we investigate algorithms to enable viewpoint-free photography for virtual reality (VR) from casual capture, i.e., from footage easily captured with consumer cameras. We build on an extensive body of work in image-based rendering (IBR). Given images of an object or scene, IBR methods aim to predict the appearance of an image taken from a novel perspective. Most IBR methods focus on full or near-interpolation, where the output viewpoints either lie directly between captured images, or nearby. These methods are not suitable for VR, where the user has significant range of motion and can look in all directions. Thus, it is essential to create viewpoint-free photos with a wide field-of-view and sufficient positional freedom to cover the range of motion a user might experience in VR. We focus on two VR experiences: 1) Seated VR experiences, where the user can lean in different directions. This simplifies the problem, as the scene is only observed from a small range of viewpoints. Thus, we focus on easy capture, showing how to turn panorama-style capture into 3D photos, a simple representation for viewpoint-free photos, and also how to speed up processing so users can see the final result on-site. 2) Room-scale VR experiences, where the user can explore vastly different perspectives. This is challenging: More input footage is needed, maintaining real-time display rates becomes difficult, view-dependent appearance and object backsides need to be modelled, all while preventing noticeable mistakes. We address these challenges by: (1) creating refined geometry for each input photograph, (2) using a fast tiled rendering algorithm to achieve real-time display rates, and (3) using a convolutional neural network to hide visual mistakes during compositing. Overall, we provide evidence that viewpoint-free photography is feasible from casual capture. We thoroughly compare with the state-of-the-art, showing that our methods achieve both a numerical improvement and a clear increase in visual quality for both seated and room-scale VR experiences

Weighted Minimal Hypersurface Reconstruction

Many problems in computer vision can be formulated as a minimization problem for an energy functional. If this functional is given as an integral of a scalar-valued weight function over an unknown hypersurface, then the sought-after minimal surface can be determined as a solution of the functional's Euler-Lagrange equation. This paper deals with a general class of weight functions that may depend on surface point coordinates as well as surface orientation. We derive the Euler-Lagrange equation in arbitrary dimensional space without the need for any surface parameterization, generalizing existing proofs. Our work opens up the possibility of solving problems involving minimal hypersurfaces in a dimension higher than three, which were previously impossible to solve in practice. We also introduce two applications of our new framework: We show how to reconstruct temporally coherent geometry from multiple video streams, and we use the same framework for the volumetric reconstruction of refractive and transparent natural phenomena, here bodies of flowing water

Weighted Patch-Based Reconstruction: Linking (Multi-view) Stereo to Scale Space

Surface reconstruction using patch-based multi-view stereo commonly assumes that the underlying surface is locally planar. This is typically not true so that least-squares fitting of a planar patch leads to systematic errors which are of particular importance for multi-scale surface reconstruction. In a recent paper, we determined the modulation transfer function of a classical patch-based stereo system. Our key insight was that the reconstructed surface is a box-filtered version of the original surface. Since the box filter is not a true low-pass filter this causes high-frequency artifacts In this paper, we propose an extended reconstruction model by weighting the least-squares fit of the 3D patch. We show that if the weighting function meets specified criteria the reconstructed surface is the convolution of the original surface with that weighting function. A choice of particular interest is the Gaussian which is commonly used in image and signal processing but left unexploited by many multi-view stereo algorithms. Finally, we demonstrate the effects of our theoretic findings using experiments on synthetic and real-world data sets

Quantitative modeling of the annealing-induced changes of the magnetotransport in Ga1−xMnxAs alloys

We study the changes of magnetoresistance induced by controlled thermal annealing at temperatures ranging from 300 to 600 °C of a Ga0.98Mn0.02As alloy grown by low-temperature molecular beam epitaxy. We use a resistor-network model for describing the electrical transport as a function of temperature and external magnetic field. The model is founded on classical semiconductor band transport and neglects many-body interactions. The peculiarities of dilute magnetic semiconductors, in particular, the magnetic-field induced changes of the density of states and the potential fluctuations due to the giant Zeeman splitting in the paramagnetic phase as well as spontaneous magnetization effects in the ferromagnetic phase, are accounted for in a mean-field fashion. This empirical transport model based on reasonable assumptions and realistic material parameters yields a satisfactory quantitative description of the experimentally obtained temperature and magnetic-field dependence of the resistivity of the entire series of annealed Ga0.98Mn0.02As samples, which exhibit metallic, semiconducting, and almost insulating transport behavior with increasing annealing temperature. Our analysis provides further understanding of the annealing-induced changes of the transport properties in dilute magnetic III-Mn-V semiconductors

Spin-dependent localization effects in GaAs:Mn/MnAs granular paramagnetic-ferromagnetic hybrids at low temperatures

Abstract We compare the magneto-transport in paramagnetic-ferromagnetic GaAs:Mn/MnAs granular hybrids and paramagnetic GaAs:Mn reference samples. The differences in the hole transport between the two systems at low temperatures arise due to carrier localization effects at the cluster-matrix interface in the hybrids. The localization is caused by a Schottky barrier formation at the interface as well as spin-dependent shifts of the hole bands caused by the stray field of the ferromagnetic clusters. The application of an external magnetic field leads to a delocalization of the carriers and thus a negative magneto-resistance effect. These effects can be simulated using a network model approach